VENTUNO Q: local LLM chat problems after the October 2026 board update
=====================================================================

What changed
------------
On Oct 22, 2026 the board's updater upgraded the Arduino stack:
  - arduino-app-cli 0.13.0 -> 0.14.0
  - App Lab assets/bricks 0.12.0 -> 0.13.1
  - the local LLM service (genai-llm-vlm-service) 1.3.0 -> 1.5.0

My offline chat app had worked perfectly before this. Afterwards, two things
broke. Neither was caused by the app's own code.


Problem 1: chat fails after a few long answers
----------------------------------------------
Symptom: after 2-4 long replies, the chat answers "[truncated]" or
"500 Service temporarily unavailable". From then on, every request fails
(HTTP 400) until the LLM service container is restarted.

What I found:
- It happens with LLM service 1.5.0, even with Arduino's own Edge AI
  Assistant example prompt.
- Short replies never trigger it. Once one request plus its reply passes
  roughly 7,000 characters, the NEXT request fails.
- Retrying, deleting the completion, and opening fresh connections don't
  help. Only restarting the service clears it.
- The old service, 1.3.0, passed every test that broke 1.5.0, including
  8 long answers in a row through the current LLM brick.

Workaround: pin the LLM service back to 1.3.0. The included script
tools/pin-llm-service.sh rewrites the genie service file in
/var/lib/arduino-app-cli/assets/<version>/services/arduino/genie/
using Arduino's own pre-update (0.12.0) settings, keeps a backup, and
restarts the app. No sudo is needed. Run it with --undo to put Arduino's
version back. A future Arduino update will probably undo the pin; just run
the script again.


Problem 2: changes to the system prompt are ignored
---------------------------------------------------
Symptom: my effort levels (Quick / Balanced / Thorough), custom
instructions and user profile stopped having any effect. Different settings
gave word-for-word identical answers. Even adding "Answer only in French"
gave an English answer.

What I found: if the new system prompt STARTS the same way as the previous
one, the service keeps using the old one. My app added the changing parts
at the END of a long base prompt, so changes were never picked up.

Fix: put the parts that change (effort, instructions, profile) at the START
of the system prompt. After that, Quick gave a 23-word answer, Thorough
gave 299 words, and "Answer only in French" replied in French.


Also useful
-----------
- The model's real context window is 4,096 tokens (system prompt, chat
  history and reply combined). Keep system prompts short.
- Reply length limits are hard stops. The 4B model often writes more than
  asked, so set the limit well above what the prompt asks for, or replies
  get cut off mid-sentence.
- To check which LLM service your board is running:
    docker ps --format '{{.Names}}  {{.Image}}' | grep genie

Tested on a VENTUNO Q with the Qwen3-4B-Instruct (genie) model, October 2026.
The attached "llm and vision multi program" zip includes both fixes.
